/home/runner/work/strom/strom/.venv/lib/python3.12/site-packages/sklearn/svm/_classes.py:31: FutureWarning:
The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning.
/home/runner/work/strom/strom/.venv/lib/python3.12/site-packages/sklearn/svm/_base.py:1237: ConvergenceWarning:
Liblinear failed to converge, increase the number of iterations.
Model Cards provide a framework for transparent, responsible reporting.
Use the vetiver `.qmd` Quarto template as a place to start,
with vetiver.model_card()
Writing pin:
Name: 'wd-svm'
Version: 20251121T090727Z-c449bSVM
Before moving forward with the to-do list, let’s throw a Random Forest to it.
SVM
For many reasons, Random Forest is usually a very good baseline model. In this particular case I started with the polynomial OLS as baseline model, just because it was so evident from the correlations that the relationship between temperature and consumption follows a polynomial shape. But let’s go back to a beloved RF.
⏩ stepit 'svm_raw': Starting execution of `strom.modelling.assess_model()` 2025-11-21 09:07:28 /home/runner/work/strom/strom/.venv/lib/python3.12/site-packages/sklearn/svm/_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning. /home/runner/work/strom/strom/.venv/lib/python3.12/site-packages/sklearn/svm/_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations. ⏩ stepit 'get_single_split_metrics': Starting execution of `strom.modelling.get_single_split_metrics()` 2025-11-21 09:07:28 ✅ stepit 'get_single_split_metrics': Successfully completed and cached [exec time 0.0 seconds, cache time 0.0 seconds, size 1.0 KB] `strom.modelling.get_single_split_metrics()` 2025-11-21 09:07:28 ♻️ stepit 'cross_validate_pipe': is up-to-date. Using cached result for `strom.modelling.cross_validate_pipe()` 2025-11-21 09:07:28 ✅ stepit 'svm_raw': Successfully completed and cached [exec time 0.1 seconds, cache time 0.0 seconds, size 14.8 KB] `strom.modelling.assess_model()` 2025-11-21 09:07:28
Metrics
| Single Split | CV | |||
|---|---|---|---|---|
| train | test | test | train | |
| MAE - Mean Absolute Error | 2.759698 | 2.681662 | 3.087172 | 3.141476 |
| MSE - Mean Squared Error | 17.777598 | 22.406892 | 15.989639 | 21.179925 |
| RMSE - Root Mean Squared Error | 4.216349 | 4.733592 | 3.781907 | 4.580536 |
| R2 - Coefficient of Determination | 0.815390 | 0.751085 | -7.118546 | 0.784520 |
| MAPE - Mean Absolute Percentage Error | 0.311556 | 0.363330 | 0.660358 | 0.254285 |
| EVS - Explained Variance Score | 0.818426 | 0.751110 | -1.855572 | 0.824080 |
| MeAE - Median Absolute Error | 2.164015 | 1.940085 | 2.542521 | 2.327501 |
| D2 - D2 Absolute Error Score | 0.612945 | 0.605302 | -1.968504 | 0.553734 |
| Pinball - Mean Pinball Loss | 1.379849 | 1.340831 | 1.543586 | 1.570738 |
Scatter plot matrix
Observed vs. Predicted and Residuals vs. Predicted
Check for …
check the residuals to assess the goodness of fit.
- white noise or is there a pattern?
- heteroscedasticity?
- non-linearity?
Normality of Residuals:
Check for …
- Are residuals normally distributed?
Leverage
Scale-Location plot
Residuals Autocorrelation Plot
Residuals vs Time
Well, not that bad, but it is overfitting quite a lot.
♻️ stepit 'grid_search_pipe': is up-to-date. Using cached result for `strom.modelling.grid_search_pipe()` 2025-11-21 09:07:31
Model Cards provide a framework for transparent, responsible reporting.
Use the vetiver `.qmd` Quarto template as a place to start,
with vetiver.model_card()
Writing pin:
Name: 'wd-svm'
Version: 20251121T090731Z-dd803
⏩ stepit 'svm_tuned': Starting execution of `strom.modelling.assess_model()` 2025-11-21 09:07:31 /home/runner/work/strom/strom/.venv/lib/python3.12/site-packages/sklearn/svm/_classes.py:31: FutureWarning: The default value of `dual` will change from `True` to `'auto'` in 1.5. Set the value of `dual` explicitly to suppress the warning. /home/runner/work/strom/strom/.venv/lib/python3.12/site-packages/sklearn/svm/_base.py:1237: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations. ⏩ stepit 'get_single_split_metrics': Starting execution of `strom.modelling.get_single_split_metrics()` 2025-11-21 09:07:31 ✅ stepit 'get_single_split_metrics': Successfully completed and cached [exec time 0.0 seconds, cache time 0.0 seconds, size 1.0 KB] `strom.modelling.get_single_split_metrics()` 2025-11-21 09:07:31 ♻️ stepit 'cross_validate_pipe': is up-to-date. Using cached result for `strom.modelling.cross_validate_pipe()` 2025-11-21 09:07:31 ✅ stepit 'svm_tuned': Successfully completed and cached [exec time 0.1 seconds, cache time 0.0 seconds, size 14.7 KB] `strom.modelling.assess_model()` 2025-11-21 09:07:31
Metrics
| Single Split | CV | |||
|---|---|---|---|---|
| train | test | test | train | |
| MAE - Mean Absolute Error | 2.341431 | 2.154123 | 2.147532 | 2.581758 |
| MSE - Mean Squared Error | 15.653589 | 20.771099 | 7.877316 | 17.898406 |
| RMSE - Root Mean Squared Error | 3.956462 | 4.557532 | 2.711544 | 4.228775 |
| R2 - Coefficient of Determination | 0.837447 | 0.769257 | -1.935593 | 0.817129 |
| MAPE - Mean Absolute Percentage Error | 0.184828 | 0.183432 | 0.474445 | 0.178073 |
| EVS - Explained Variance Score | 0.839705 | 0.788348 | -1.089626 | 0.820615 |
| MeAE - Median Absolute Error | 1.497370 | 1.098866 | 1.790379 | 1.581437 |
| D2 - D2 Absolute Error Score | 0.671608 | 0.682947 | -0.894147 | 0.631738 |
| Pinball - Mean Pinball Loss | 1.170716 | 1.077061 | 1.073766 | 1.290879 |
Scatter plot matrix
Observed vs. Predicted and Residuals vs. Predicted
Check for …
check the residuals to assess the goodness of fit.
- white noise or is there a pattern?
- heteroscedasticity?
- non-linearity?
Normality of Residuals:
Check for …
- Are residuals normally distributed?
Leverage
Scale-Location plot
Residuals Autocorrelation Plot
